SPECIFICATION 

TITLE OF INVENTION 

APPARATUS AND METHOD FOR IMAGE PROCESSING AND STORAGE 

MEDIUM FOR THE SAME 

5 BACKGROUND OF THE INVENTION 

The present invention relates to an apparatus and a method for image 
processing as well as a storage medium for the same, and particularly to an apparatus 
and a method for image processing as well as a storage medium that are preferably 
used for transforming a dynamic range of intensity levels of pixels forming an image. 
10 As the performance of CCDs (Charge Coupled Devices) installed in electronic 

apparatuses that have the ability to acquire image data, such as digital cameras, 
scanners and the like, has become higher, a gradation dynamic range of images tends 
to become wider. 

However, when images having a wider gradation dynamic range than 
15 conventional one (hereinafter referred to as "wider dynamic range images'') are 
supplied to reproducing apparatuses such as displays, printers and the like, which 
correspond to images having conventional dynamic range (hereinafter referred to as 
"narrower dynamic range images"), there is a need for a gradation correction technique 
that transforms the wider dynamic range images into the narrower dynamic range 
20 images. 

As one of conventional gradation correction techniques, a histogram 
equalization method is well known. The histogram equalization method transforms 
pixels' intensity levels so that the intensity levels are distributed uniformly over an 
entire image. 

25 A basic algorithm for the histogram equalization method will be now 

described. As shown in Figure lA, a histogram showing a frequency of every 
intensity level for an input image (an image before transformation), a histogram 
showing the number of pixels having identical intensity levels is generated. Assuming 
that a frequency of an intensity level Xn is H(Xn) (n = 0, 1, 2,..., max), the histogram 

30 thereof is cumulated. More specifically, a cimiulative frequency H(Xo) for an 
intensity level Xq is set to H(Xo); a cumulative frequency H(Xi) for an intensity level 



Xi is obtained by calculation of [H(Xo) + H(Xi)]; a cumulative frequency H(X2) for 
an intensity level X2 is obtained by calculation of P(Xo) + H(Xi) + H(X2)]; and a 
cumulative frequency H(Xmax) for an intensity level X^ax is obtained by calculation 
of [H(Xo) + H(Xi) + ... + H(Xmax)]* As a result of such cumulation, a cumulative 
5 histogram is generated. Further, a look up table (hereinafter referred to as "LUT") 
shown in Figure IB is generated by adapting a dynamic range (X'o - X'max) of 
intensity levels of an output image to a range from the cumulative frequency H(Xo) 
for the intensity level Xo to the cumulative frequency H(Xmax) for the intensity level 
Xmax (the vertical axis of the cumulative histogram). 

10 When the intensity levels of the input image are transformed using the LUT 

generated in this manner, the resultant output image has a uniform frequency (an even 
distribution) of its intensity levels as shown in a histogram of Figure 1 C and therefore 
has a more enhanced contrast. 

However, when such histogram equalization method is adapted to an input 

1 5 image that has an extreme peak of its intensity frequency, intensity levels near the peak 
may be transformed into a wider range within the dynamic range of a resultant output 
image, and therefore a desired result may not be obtained (the resultant image may be 
indistinct). 

In addition, in the conventional gradation correction technique, since human 
20 visual characteristics, that is, more particularly, Weber-Fechner's law stating that 
"human sensation is proportional to the logarithm of stimulus intensity" are not 
considered, the narrower dynamic range image as a result of the transformation may be 
indistinct. 

SUMMARY OF THE INVENTION 

25 An object of the present invention is, therefore, to provide intensity 

transformation that can be performed considering both intensity distribution 
characteristics of input images and the human visual characteristics. 

To achieve the above object, according a first embodiment of the present 
invention, there is provided an image processing apparatus for correcting a gradation 

30 of an input image data, including: a first generating section for generating a first look 
up table using the input image data; a second generating section for generating a 
second look up table based upon the first look up table and human visual 



characteristics; a combining section for generating a third look up table by combining 
the first and second look up tables according to a predetermined combining ratio; and a 
transforming section for transforming the image data using the third look up table. 

In an embodiment, the first generating section may generate the first look up 
5 table based upon a histogram of intensity levels of the image data. 

In an embodiment, the second generating section may generate the second look 
up table using a predetermined logarithm curve as the hxnnan visual characteristics. 

In an embodiment, the image processing apparatus, preferably, farther includes 
an input section for inputting the combining ratio. 
10 In an embodiment, the image processing apparatus, preferably, further includes 

a setting section for setting the combining ratio based upon the first and second look 
up tables. 

In an embodiment, the transforming section may transform a dynamic range of 
the intensity levels of the image data using the third look up table. 

15 According to a second embodiment of the present invention, there is provided 

an image processing method, wherein an image processing apparatus is used for 
correcting a gradation of an input image data, the method including the steps of 
generating a first look up table using the input image data, generating a second look up 
table based upon the first look up table and human visual characteristics, generating a 

20 third look up table by combining the first and second look up tables according to a 
predetermined combining ratio, and transforming the image data using the third look 
up table. 

According to a third embodiment of the present invention, there is provided a 
storage medium in which a computer readable program for image processing to correct 

25 a gradation of an input image data, the program including the steps of generating a first 
look up table using the input image data, generating a second look up table based upon 
the first look up table and human visual characteristics; generating a third look up table 
by combining the first and second look up table according to a predetermined 
combining ratio, and transforming the image data using the third look up table. 

30 According to a fourth embodiment of the present invention, there is provided 

an image processing apparatus for correcting a gradation of an input image data, the 
apparatus including a first generating section for generating a first look up table using 
the input image data, a second generating section for generating a second look up table 



based upon the first look up table and human visual characteristics, and a transforming 
section for transforming the image data using the second look up table. 

In an embodiment, the first generating section may generate the first look up 
table based upon a histogram of intensity levels of the image data. 
5 In an embodiment, the second generating section may generate the second look 

up table using a predetermined logarithm curve as the human visual characteristics. 

In an embodiment, the transforming section may transform a dynamic range of 
the intensity levels of the image data using the second look up table. 

According to a fifth embodiment of the present invention, there is provided an 

10 image processing method of an image processing apparatus for correcting a gradation 
of an input image data, the method including the steps of generating a first look up 
table using the input image data, generating a second look up table based upon the first 
look up table and human visual characteristics, and transforming the image data using 
the second look up table. 

15 According to a sixth embodiment of the present invention, there is provided a 

storage medium in which a computer readable program for image processing to correct 
a gradation of an input image data, the program including the steps of generating a first 
look up table using the input image data, generating a second look up table based upon 
the first look up table and human visual characteristics, and transforming the image 

20 data using the second look up table. 

In the image processing apparatus and method as w^ell as the program 
according to the first, second, and third embodiment of the present invention, a first 
look up table that has been generated using input image data and a second look up 
table that has been generated based upon the first look up table and human visual 

25 characteristics are combined according to a predetermined combining ratio to generate 
a third look up table, and then the image data is transformed using the third look up 
table, so that it is possible to transform intensity levels with consideration of both the 
features of distribution of the intensity levels in the input image and the human visual 
characteristics. 

30 Further, in the image processing apparatus and method as well as the program 

according to the fourth, fifth, and sixth embodiments of the present invention, a first 
look up table is generated using input image data, a second look up table is generated 
based upon the first look up table and human visual characteristics, and then the image 



data is transfomed using the second look up table, so that it is possible to transform 
intensity levels with consideration of both the features of distribution of the intensity 
levels in the input image and the human visual characteristics. 

Additional features and advantages of the present invention are described in, 
5 and will be apparent from, the following Detailed Description of the Invention and the 
Figures. 

BRIEF DESCRIPTION OF THE FIGURES 

Figures lA through IC show diagrams illustrating a histogram equalization 
method. 

10 Figure 2 shows a block diagram of an imaging apparatus in accordance with 

the principles of the present invention. 

Figure 3 shows a block diagram of an image processing section in accordance 
with the principles of the present invention. 

Figure 4 shows a diagram illustrating Weber-Fechner's law. 
15 Figure 5 shows a flow chart describing an operation of an image processing 

section in accordance with the principles of the present invention. 

Figure 6 shows a flow chart illustrating a process for generating a visual 
characteristics LUT in Step S4 of Figure 5. 

Figure 7 shows a diagram illustrating a process for generating a combined 

20 LUT. 

Figure 8 shows a block diagram of an image processing section in accordance 
with the principles of the present invention. 

Figure 9 shows a block diagram of an image processing section in accordance 
with the principles of the present invention. 
25 DETAILED DESCRIPTION OF THE INVENTION 

Figure 2 shows an imaging apparatus in accordance with an embodiment of the 
present invention. Preferably, the imaging apparatus acquires an optical image of an 
object as wider dynamic range (e.g., 16-bit wide) image data and transforms it into 
narrower dynamic range (e.g., 8-bit wide) image data appropriately. 
30 In an embodiment, an optical system of the imaging apparatus includes: a lens 

1 that collects the optical image of the object; an aperture 2 that adjusts a quantity of 
light of the optical image; a shutter 3 that adjusts duration of incidence of the optical 

5 



image; an infrared cutoff filter (IR filter) 4 that filters out light in infrared region to 
which a CCD 7 is too sensitive; and a low pass filter (LPF) 5 that rejects higher optical 
frequency components to suppress aliasing as a result of discrete sampling by the CCD 
7. It should be noted that the infrared cutoff filter 4 may be omitted by applying a 
5 coating for rejecting the infrared region to the lens 1 . 

Further, the imaging apparatus includes: a timing generator 6 that generates 
horizontal scanning timing and vertical scanning timing for the CCD 7; the CCD 7 that 
outputs wider dynamic range image data to a CDS 8 by photoelectrically converting 
the optical image input via the optical system; the CDS (correlated double sampling) 8 

10 that reduces noise by sampling the image data input from the CCD 7; an AGC 
(automatic gain controller) 9 that electrically amplifies the amplitude of the image data 
depending on the brightness of the object; an AID converter 10 that digitizes the 
analog image data; an image processing section 1 1 that comprises a DSP (digital signal 
processor) and so on and converts the wider dynamic range digital image data into 

15 narrower dynamic range digital image data; and a controlling section 12 that controls a 
drive 14 to read a controlling program stored in a magnetic disk 15, an optical disk 16, 
a magneto-optical disk, or a semiconductor memory 18 and further controls the entire 
imaging apparatus based upon the controlling program read through the drive 14, user 
commands input through a manipulating section 13 and the like. 

20 In the imaging apparatus, the optical image that is input via the optical system 

including a series of components from the lens 1 to the low pass filter 5 is converted 
into the wider dynamic range image data by the CCD 7, imdergoes the noise reduction 
process by the CDS 8 and the amplitude amplification by the AGC 9, and is further 
digitized by the AID converter 10 and input into the image processing section 1 L The 

25 Wider dynamic range digital image data that is input into the image processing section 
1 1 is transformed into the narrower dynamic range digital image data. 

Figure 3 shows an embodiment of image processing section 1 1 (see Figure 2). 
Preferably, an image memory 31 stores the wider dynamic range digital image data 
input from the AID converter 10. A histogram generating section 32 generates a 

30 histogram showing distribution of intensity levels in the image data read from the 
image memory 31 and stores it in a histogram memory 33. A histogram equalization 
LUT generating section 34 calculates a cumulative histogram by cumulating the 
histogram read from the histogram memory 33 in a manner similar to the process for 



generating a LUT of Figure IB from a histogram of Figure lA, and further adapts a 
narrower dynamic range to the vertical axis of the cumulative histogram in order to 
generate a histogram equalization LUT and store it in the histogram memory 33. 

A visual characteristics LUT generating section 35, referring to the histogram 
5 equalization LUT stored in the histogram memory 33, generates another LUT with 
consideration of human visual characteristics (hereinafter referred to as a "visual 
characteristics LUT") and stores it in a visual characteristics memory 36. The human 
visual characteristics and the process for generating the visual characteristics LUT vdll 
be described in detail later. 

10 A LUT combining section 37 combines the histogram equalization LUT read 

from the histogram memory 33 and the visual characteristics LUT read from the visual 
characteristics LUT memory 36 according to a combining ratio read from a combining 
ratio memory 38 and stores the resultant combined LUT in a combined LUT memory 
39. Preferably, the combining ratio memory 38 stores the combining ratio that has 

1 5 been set by the user using the manipulating section 1 3 . 

An image data correction section 40 transforms intensity levels of the wider 
dynamic range image data read ftom the image memory 3 1 using the combined LUT 
read from the combined LUT memory 39 and outputs the resultant narrower dynamic 
range image data as corrected data. 

20 Figure 4 shows a diagram illustrating Weber-Fechner's law. Weber-Fechner's 

law states that "human sensation is proportional to the logarithm of stimulus intensity." 
That is, assuming that the stimulus intensity is R and the sensation is E, the following 
relational expression is established: 
Sensation E = K * logR 

25 where K is a predetermined constant. 

Figure 5 shows a flow chart describing an operation of an imaging processing 
section 1 1 (see Figure 3) in accordance with an embodiment of the present invention. 
In Step SI, wider dynamic range digital image data is input from the A/D converter 10 
(see Figvire 2) to the image processing section 1 1 . The input image data is stored in 
30 the image memory 31. In Step S2, the histogram generating section 32 reads the 
image data from the image memory 31 and generates a histogram of the image data's 
intensity levels and stores it in the histogram memory 33. In Step S3, the histogram 



equalization LUT generating section 34 reads the histogram from the histogram 
memory 33 to calculate a cumulative LUT, and then generates a histogram 
equalization LUT by adapting a narrower dynamic range to the vertical axis of the 
cumulative histogram and stores it in the histogram memory 33. 
5 In Step S4, the visual characteristics LUT generating section 35 generates a 

visual characteristics LUT by referring to the histogram equalization LUT stored in the 
histogram memory 33 and stores it in the visual characteristics memory 36. A process 
for generating the visual characteristics LUT will be described with reference to the 
flow chart of Figure 6 as well as Figure 7. It is noted that Figure 7 shows a curve for 

10 the histogram equalization LUT, a curve for the visual characteristics LUT and a curve 
for the combined LUT. 

In Step SUj the visiial characteristics LUT generating section 35 sets two 
thresholds Nl, N2 for determination higher and lower noise levels for the narrower 
dynamic range. The thresholds may be set, for example, based upon the shape of the 

15 curve of the histogram equalization LUT (the cumulative histogram), or it may be set 
to any predetermined fixed value, or it may be also set by the user. 

In Step SI 2, the visual characteristics LUT generating section 35 determines 
for the histogram equalization LUT the minimum value "low" that is the lowest value 
in the wider dynamic range that corresponds to the values higher than the threshold for 

20 the lower noise level Nl in the narrower dynamic range, and determines a point which 
value is "low" in the wider dynamic range and the minimum ("zero") in the narrower 
dynamic range as PL 

In Step SI 3, the visual characteristics LUT generating section 35 determines 
for the histogram equalization LUT the maximum value "high" that is the highest 

25 value in the wider dynamic range that corresponds to the values lower than the 
threshold for the higher noise level N2 in the narrower dj^amic range, and determines 
a point which value is "high" in the wider dynamic range and the maximum ("max") in 
the narrower dynamic range as P2. 

In Step SI 4, the visual characteristics LUT generating section 35 calculates a 

30 logarithm curve y that passes through the points PI, P2 as the visual characteristics 
LUT: 

y = a * log(x) + b 
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where x, y designate values of the input intensity (the horizontal axis) and the output 
intensity (the vertical axis), respectively. 

Nov^, referring back to Figure 5, in Step S5, the LUT combining section 37 
reads the histogram equalization LUT (hereinafter also referred to as LUTl(x)) from 
5 the histogram memory 33, reads the visual characteristics LUT (hereinafter also 
referred to as LUT2(x)) from the visual characteristics LUT memory 36, reads the 
combining ratio from the combining ratio memory 38, and then calculates a combined 
LUT (hereinafter also referred to as LUT3(x)) according to the following expressions 
and stores it in the combined LUT memory 39: 
10 LUT3(x) = 0 (0 < X < low) 

LUT3(x) = * LUTl(x) + (1 - ) * LUT2(x) (low<x<high) 
LUT3(x) = max (high <x< max) 

In Step S6, the image data correcting section 40 reads the wider dynamic range 
digital image data from the image memory 31, transforms the intensity levels of the 
15 wider dynamic range digital image data using the combined LUT read from the 
combined LUT memory 39 and outputs the resultant narrower dynamic range image 
data as the corrected data. 

As described above, in the image processing section 11, the wider dynamic 
range digital data is transformed into the narrower dynamic range digital image data by 
20 using the combined LUT into which the histogram equalization LUT based upon the 
histogram of the intensity levels of the wider dynamic range image data and the visual 
characteristics LUT with consideration of the human visual characteristics are 
combined. 

Figure 8 shows another embodiment image processing section 1 1 . Preferably, 
25 a combining ratio computing section 51 has been added to the image processing 
section 11, wherein the combining ratio computing section 51 computes the combining 
ratio by analyzing features of the histogram equalization LUT read from the histogram 
memory 33 and the visual characteristics LUT read from the visual characteristics 
LUT memory 36 and stores it in the combining ratio memory 38. 
30 The operation of this embodiment of the image processing section 1 1 is similar 

to the image processing station shown in Figure 3, except that a process for computing 
the combining ratio by the combining ratio computing section 51 is added to the 
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process described with reference to the flow chart of Figure 5 between the process of 
Step 4 (the process for generating the visual characteristics LUT) and the process of 
Step 5 (the process for combining both LUTs), and therefore its description is omitted. 
Figure 9 shows another embodiment of the image processing section 11. 
5 Preferably, the LUT combining section 37, the combining ratio memory 38, and the 
combined LUT memory 39 have been deleted from the image processing section 
shown in Figure 3, 

Preferably, the image correcting section 40 transforms the intensity levels of 
the wider dynamic range image data using the visual characteristics LUT read from the 
10 visual characteristics LUT memory 36 to acquire the narrower dynamic range image 
data. 

In an embodiment, it is also possible to apply the present invention to the case 
in which the intensity levels are altered without alteration of the dynamic range of 
images. 

15 In an embodiment, it is also possible to apply the present invention not only to 

the imaging apparatus in the embodiments described above, but also to other electronic 
apparatuses that process image data, such as scanners, facsimile machines, copying 
machines and the like. 

A series of the above-described processes may be performed by hardware, 

20 though it also may be performed by software in further embodiments of the present 
invention. Preferably, when the series of the processes is performed by software, a 
program constituting the software and stored in a storage medium shall be installed in 
a computer incorporated in dedicated hardware, or in any other computer that can 
perform various ftmctions by installing various programs, such as, for example, a 

25 general-purpose personal computer and the like. 

The storage medium may, as shown in Figure 2, include package media that 
may include magnetic disks 15 (including floppy disks), optical disks 16 (including 
CD-ROM (compact disk read only memory) and DVD (digital versatile disks)), 
magneto-optical disks 17 (including MD (Mini Discs)), or semiconductor memory 18 

30 and the like, in which the program is stored and which are supplied to provide the user 
with the program, or otherwise it may include ROM or hard disk drives, in which the 
program is stored and which is supplied to the user, being pre-installed in the 
computer. 



In this specification, it should be noted that the steps describing the program 
stored in the storage medium may include not only the processes that are performed 
chronologically in the order in which they have been described, but also the processes 
that are performed not always chronologically, but in parallel or independently. 

Although the present invention has been described with reference to specific 
embodiments, those of skill in the art will recognize that changes may be made thereto 
without departing firom the spirit and scope of the invention as set forth in the hereafter 
appended claims. 
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